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Abstract 


A  language  called  DMPL  for  programming  distributed  real-time,  nrixed-criticality  software  is 
presented.  DMPL  supports  a  distributed  system  in  which  each  node  executes  a  set  of  peri¬ 
odic  real-time  threads  that  are  scheduled  on  the  basis  of  their  priority  and  criticality.  Both 
synchronous  and  asynchronous  threads  are  allowed.  The  syntax  and  semantics  of  DMPL  are  for¬ 
mally  described.  A  compiler  that  generates  c-| — I-  code  from  a  DMPL  program  is  presented.  Two 
methods  of  verification  of  properties  of  synchronous  threads  via  sequentialization  are  proposed: 
fully-automated  bounded  model  checking,  and  deductive  verification  with  manually-supplied 
invariants.  DMPL  programming  and  verification  are  validated  on  several  examples  of  collision 
avoidance  in  multi-robot  systems. 
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1  Introduction 


Development  of  verified  distributed  real-time  software  (DRTS)  is  becoming  increasingly  impor¬ 
tant.  A  major  driver  is  the  movement  of  safety-critical  real-time  embedded  domains — e.g.,  au¬ 
tomotive,  avionics,  medical  devices,  and  robotics — toward  more  distributed  deployments  where 
multiple  physically  separated  nodes  communicate  and  coordinate  to  provide  increased  capability, 
resilience  to  cyber-attacks,  fault-tolerance  etc.  In  other  words,  the  traditional  “air  gap”  between 
real-time  embedded  devices  is  disappearing.  For  example,  there  is  a  push  toward  autonomous 
vehicles,  and  intersection  protocols  [1]  to  reduce  accidents.  Robots  are  being  deployed  in  teams 
to  achieve  more  complex  tasks  with  increased  fault-tolerance  [20].  Even  medical  devices  are 
being  connected  in  a  plug-and-play  manner  for  critical  procedures  [33] . 

DRTS  brings  together  two  software  domains — distributed  and  real  time — each  with  a  rich  history 
for  development,  validation,  and  certification.  However,  they  have  been  studied  and  developed 
by  different  communities  of  researchers  and  engineers.  Consequently,  programming  languages 
have  specialized  for  each  domain.  For  example,  languages  for  distributed  systems  focus  on 
parallelization  and  orchestration  of  tasks  and  their  results,  and  average  case  performance.  Dis¬ 
tributed  software  is  developed  using  a  wide  variety  of  languages,  and  thread  scheduling  to  meet 
hard  real-time  deadlines  is  not  a  first  class  concern.  In  contrast,  real-time  software  is  written 
largely  in  C/C-l — |-  and  executed  on  real-time  operating  systems,  with  explicit  control  on  thread 
scheduling  and  worst-case  response  times  to  meet  hard  deadlines  in  a  predictable  way.  The 
semantics  of  communication  between  nodes  is  poorly  specified  and  dependent  on  middleware. 
Neither  approach  is  sufficient  for  producing  verified  DRTS. 

Consequently,  DRTS  is  developed  today  using  languages  and  tools  that  neither  have  the  right 
concepts  and  abstractions  nor  support  rigorous  verification.  In  this  technical  report,  we  ad¬ 
dress  this  challenge  via  a  new  domain-specific  programming  language,  DMPL,  for  DRTS.  DMPL 
combines  concepts  from  both  disciplines — real-time  via  thread-scheduling  and  distributed  via 
inter-thread  and  inter-node  communication  semantics-  in  a  precise  way  and  supports  specifica¬ 
tion  and  verification  of  safety  properties. 

Roles.  A  DMPL  program  specifies  a  set  of  roles,  each  consisting  of  a  set  of  real-time  threads  that 
communicate  via  shared  variables.  At  runtime,  each  node  of  the  DRTS  executes  a  specific  role. 
Each  thread  executes  an  infinite  sequence  of  jobs  (semantically,  each  job  is  a  procedure)  starting 
at  regular  time  intervals  (its  period).  Each  node  has  its  own  thread  scheduler.  DMPL  supports 
mixed-criticality  scheduling  [3] ,  a  form  of  real-time  scheduling  that  allows  threads  which  perform 
tasks  with  different  levels  of  importance  (e.g.,  engine  control  vs.  multimedia  player)  to  share  a 
single  CPU  under  normal  and  overload  operating  conditions. 

Thread  Scheduling.  In  particular,  DMPL  uses  Zero-Slack  Rate  Monotonic  (zsrm)  scheduling. 
Specifically,  each  thread  is  statically  assigned  a  priority  (using  Rate- Monotonic  assignment,  i.e., 
threads  with  shorter  period  have  higher  priority)  and  a  criticality  (based  on  the  importance  of 
the  task  performed  by  the  thread).  We  assume  that  all  threads  are  ZSRM  schedulable,  i.e.,:  (i) 
under  normal  conditions,  all  jobs  complete  before  their  deadline  (i.e.,  arrival  time  of  the  next 
job);  (ii)  under  overload  conditions,  low-criticality  jobs  are  suspended  if  necessary  to  enable  high- 
criticality  jobs  to  finish  before  their  deadlines.  In  practice,  this  asymmetric  temporal  protection 
is  implemented  by  a  combination  of  a  static  schedulability  analysis  and  a  runtime  scheduler. 
The  details  of  ZSRM  are  available  elsewhere  [9]  and  are  beyond  the  scope  of  this  paper. 

Communication.  DMPL  threads  (within  and  across  nodes)  communicate  via  shared  vari¬ 
ables.  The  state  space  explosion  resulting  from  variable-based  interaction  between  threads 
is  a  major  source  of  complexity  for  both  programming  and  verification.  DMPL  uses  two 
complexity-reduction  mechanisms  to  ameliorate  this  problem.  First,  each  job  has  a  transac- 
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tional  semantics — on  arrival,  the  job  reads  all  shared  variables  atomically  into  local  copies,  then 
operates  on  these  local  copies  to  compute  new  values,  and  finally  writes  the  new  values  back 
atomically  before  terminating.  This  “read-execute-write”  behavior  reduces  inter-tlrread  interac¬ 
tion  and  enables  us  to  abstract  away  details  of  thread  scheduling  and  network  delays  to  better 
understand  the  behavior  of  the  program.  It  is  implemented  via  locks  supported  by  the  ZSRM 
scheduler. 

Synchronization.  Second,  DMPL  supports  two  kinds  of  threads — synchronous  and  asyn¬ 
chronous — each  executing  under  the  corresponding  model  of  computation.  Values  written  to  a 
share  variable  by  jobs  of  an  asynchronous  thread  are  observed  by  other  threads  only  under  the 
restriction  that  older  writes  cannot  be  observed  after  newer  ones — no  other  memory  consistency 
is  provided.  In  contrast,  a  synchronous  thread  executes  in  rounds.  During  each  round,  one  job 
of  the  thread  executes  on  each  node.  Writes  to  shared  variables  in  round  i  are  guaranteed  to 
be  observed  during  (and  only  during)  round  i  +  1.  This  restricted  communication  further  re¬ 
duces  the  system  state  space  and  has  been  known  to  reduce  verification  complexity  [22] .  Shared 
variables  and  synchronous  jobs  are  implemented  via  the  MADARA  [12]  middleware. 

Semantics.  When  defining  the  semantics  of  DMPL,  we  have  to  incorporate  the  effects  of  both 
ZSRM  scheduling  and  the  synchronous  model  of  computation.  Interestingly,  both  can  be  mod¬ 
eled  by  allowing  jobs  to  abort  their  transactions  non-deterministically  (i.e. ,  to  behave  like  No 
Operation  (NOP)  statements).  For  ZSRM,  a  low-priority  job  that  is  suspended  under  overload 
conditions  is  semantically  a  NOP.  In  the  case  of  the  synchronous  threads,  a  job  that  cannot 
proceed  because  other  nodes  are  yet  to  complete  the  previous  round  is  semantically  a  NOP. 
Note  that  the  transactive  nature  of  jobs  provides  a  natural  way  to  convert  them  to  NOPs.  We 
present  this  formally  later. 

Specification  and  Verification,  dmpl  also  supports  specification  of  safety  (i.e.,  reachability) 
properties  of  synchronous  threads.  Since  jobs  form  the  natural  unit  of  computation  in  dmpl 
programs,  such  properties  are  expressed  as  job  assertions:  conditions  that  must  hold  at  the 
beginning  of  each  job  of  a  synchronous  thread.  Since  dmpl  programs  are  distributed,  assertions 
in  dmpl  enable  us  to  express  conditions  over  multiple  (e.g.,  pairs  of)  nodes.  This  is  important 
to  capture  requirements  that  involve  more  than  one  node  (e.g.,  collision  avoidance).  Verification 
is  done  through  sequentialization.  In  essence,  for  the  target  synchronous  thread,  we  construct 
a  semantically  equivalent  sequential  program  which  is  then  verified  via  existing  software  verifi¬ 
cation  techniques.  Both  bounded  verification  (for  bug  finding)  and  deductive  verification  (for 
proving  correctness)  are  supported. 

Legacy  Components.  We  envision  that  dmpl  will  be  used  to  implement  the  overall  concurrency 
(threading)  and  communication  (shared  variables)  structure,  as  well  as  key  protocols  and  al¬ 
gorithms  that  must  be  verified  formally.  The  rest  of  the  system  will  be  implemented  by  using 
existing  components  and  libraries  (e.g.,  image  processing,  planning,  sensing,  and  actuation)  To 
support  this  mode  of  development,  DMPL  threads  can  invoke  external  functions  that  are  linked 
to  C++  libraries  during  compilation,  dmpl  also  supports  a  limited  set  of  C++  types  to  ease 
programming  and  interaction  with  such  external  functions.  Note,  however,  that  such  interaction 
is  restricted.  In  particular,  the  external  functions  can  only  return  a  value  that  could  be  stored 
in  a  DMPL  variable.  No  indirect  modifications  of  dmpl  variables  (e.g.,  through  pointers)  is  al¬ 
lowed.  In  our  experience,  this  provides  a  good  tradeoff  between  ease  of  development,  cleanliness 
of  semantics,  and  soundness  of  verification. 

Code  Generation  and  Validation.  We  show  how  to  generate  C++  code  from  DMPL  programs. 
We  implemented  this  code  generation  (and  sequentialization  for  verification  purposes)  in  a  com¬ 
piler  called  DMPLC.  To  validate  DMPLC,  we  ran  several  coordinated  multi-robot  mission  ex¬ 
amples  on  the  Linux  operating  system,  implemented  a  realistic  physical  environment  with  the 
V-REP  [29]  simulator,  and  also  employed  publicly-available  implementations  of  ZSRM  [35]  and 
MADARA  [21].  Our  examples  use  synchronous  threads  to  implement  a  collision  avoidance  pro¬ 
tocol  and  asynchronous  threads  to  perform  other  mission-relevant  tasks  (such  as  path  planning 
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and  self-adaptation).  We  show  that  bounded  model  checking  is  able  to  detect  bugs  in  incorrect 
versions  of  the  protocol;  deductive  verification  with  manually  supplied  invariants  proves  the 
correct  version  to  be  safe. 

Non- Goals.  Our  contribution  comes  from  the  perspective  of  programming  language  design  and 
implementation  (PLDI),  following  the  principle  that  a  well-defined  language  should  drive  de¬ 
velopment  and  verification.  We  rely  on  results  from  several  complementary  disciplines.  For 
example,  we  use  ZSRM  for  schedulability,  MADARA  for  communication  and  synchronization,  se- 
quentialization  and  software  model  checking  for  verification,  Linux  and  V-REP  for  the  platform 
and  environment,  and  C++  as  the  target  language.  Also,  we  assume  that  security,  networking, 
performance,  timing  verification  (i.e. ,  schedulability),  and  similar  issues  are  handled  separately. 
These  choices  were  guided  by  their  natural  support  for  implementing  dmpl  semantics,  public 
availability,  and  technical  familiarity.  We  do  not  claim  that  they  are  optimal  or  that  they  were 
made  by  discarding  others.  However,  they  do  lead  to  an  effective  engineering  process  for  verified 
DRTS  and  identify  key  design  decisions  that  must  be  made  for  verified  DRTS  development. 

The  rest  of  this  report  is  organized  as  follows.  In  Section  2,  we  survey  related  work.  In  Section  3, 
we  present  a  DRTS  as  a  motivating  example.  In  Section  4  we  present  the  syntax  and  semantics 
of  dmpl.  In  Section  5  we  describe  how  c-| — b  code  can  be  generated  from  a  dmpl  program.  In 
Section  6  we  present  an  approach  for  verifying  safety  properties  over  synchronous  threads  via 
sequentialization.  Finally,  in  Section  7  we  evaluate  our  approach  and  in  Section  8,  we  discuss 
areas  for  future  work  and  conclude. 
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2  Related  Work 


Real-time  systems  comprise  a  very  large  area  of  research.  Broadly,  ZSRM  [9]  falls  into  the 
class  of  preemptive  fixed-priority  scheduling.  This  line  of  work  originates  as  early  as  Rate- 
Monotonic  scheduling  [19]  with  many  subsequent  refinements  and  variations,  such  as  mutual 
exclusion  mechanisms  [26].  ZSRM  also  handles  mixed-criticality  [30],  which  is  a  more  recent 
concept  emerging  from  the  need  to  co-locate  multiple  software  functions  with  different  levels 
of  importance  on  a  single  hardware  infrastructure  to  take  advantage  of  features  such  as  more 
powerful  processors.  This  area  has  spawned  significant  research  [2,  3]. 

There  is  a  very  wide  literature  on  languages  for  concurrent  and  distributed  systems,  such  as 
CSP  [14],  CCS  [23],  and  the  7r-calculus  [24],  Most  of  these  languages  are  targeted  towards 
modeling  and  verifying  concurrent  systems  (such  as  FDR  [13]  for  CSP),  not  code  generation. 
The  Occam-7r  [31]  language  blends  CSP  and  the  7r-calculus,  and  supports  code  generation  and 
verification.  However,  it  is  not  targeted  toward  the  development  of  real-time  systems  and  its 
semantics  has  not  been  connected  to  periodic  tasks  with  hard  deadlines  or  mixed-criticality. 
SystemJ  [27]  is  aimed  at  programming  distributed  GALS  systems.  Although  threads  do  not 
have  to  be  real-time,  the  system  is  partitioned  into  multiple  asynchronous  clock  domains.  This 
means  that,  unlike  DMPL,  threads  across  clock  domains  cannot  execute  synchronous  protocols. 
Also,  communication  is  based  on  message-passing,  not  shared  variables. 

Software  verification  is  yet  another  wide  research  area  [15].  Our  work  complements  verification 
techniques  for  sequential  C  programs  since  we  reduce  our  verification  problem  to  a  sequential 
one.  Sequentialization  has  been  widely  used  for  concurrent  program  verification.  However, 
most  of  this  work  is  targeted  toward  multi-threaded  software  [18,  28,  7]  or  real-time  software  [5] 
executing  on  a  single  processor,  not  distributed  ones.  There  has  also  been  work  on  verifying 
distributed  algorithms  [16],  while  our  goal  is  to  verify  software. 

dmpl  is  inspired  by  the  language  dasl  [4]  and  its  verification,  but  generalizes  it  significantly. 
DASL  allows  only  one  synchronous  thread  per  node,  and  has  no  support  for  mixed-criticality,  or 
deductive  verification,  dmpl  supports  multiple  synchronous  and  asynchronous  mixed-criticality 
threads,  transactive  job  semantics,  and  both  bounded  and  deductive  verification,  which  are 
critical  for  real  system  development.  The  P  language  [10]  is  also  a  domain  specific  language 
developed  to  facilitate  programming  and  verification.  However,  it  targets  asynchronous  event 
driven  programs. 

We  build  on  the  work  of  Kyle  et  al.  [17]  but  differ  from  it  substantially.  While  they  presented 
dmpl  in  an  ad-hoc  manner  and  focused  on  the  specification  and  verification  of  probabilistic 
properties  via  statistical  model  checking  [34],  we  give  formal  syntax  and  semantics  of  DMPL, 
and  focus  on  the  specification  and  verification  of  non-deterministic  behavior  via  software  model 
checking. 
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3  An  Example  DRTS 


Consider  the  distributed  system  shown  in  Figure  3.1,  which  consists  of  a  fleet  of  five  flying  robots 
(or  nodes)  on  a  two-dimensional  grid.  One  node  has  the  sensors  critical  for  mission  success,  and 
plays  the  role  of  the  leader.  The  remaining  four  nodes  provide  defensive  countermeasures  and 
play  the  role  of  protector.  Their  defenses  rely  on  being  positioned  between  the  leader  node  and 
any  attackers.  For  the  mission  to  succeed,  the  leader  must  follow  a  specific  flight  path  and  reach 
a  particular  location  by  the  end  of  the  mission  time  T. 

Protectors  best  cover  the  sensor  node  by  flying  close  to  it;  however,  this  raises  the  chance  of  col¬ 
lisions.  Nodes  execute  a  collision-avoidance  protocol.  The  protocol  slows  down  the  overall  speed 
of  the  fleet.  The  degree  of  slowdown  depends  on  the  level  of  communication  and  coordination 
needed  by  the  protocol. 

We  assume  that  different  areas  present  different  levels  of  attack  (i.e. ,  hazards)  to  the  leader.  In 
particular,  the  map  is  a  10  x  10  grid.  For  the  upper  right  5x5  cells,  we  assume  high  hazards 
(between  0.5  and  0.9,  inclusive,  uniformly  distributed).  For  the  remaining  cells,  we  assume  low 
hazards  (between  0  and  0.4,  inclusive,  also  uniformly  distributed).  In  Figure  3.1,  high  hazard 
cells  are  colored  red  and  low  hazard  cells  are  colored  gray. 

The  system  has  two  formations:  tight  and  loose.  In  tight  formation,  the  protectors  are  closer  to 
the  leader  and  provide  better  protection.  However,  the  tight  formation  moves  at  about  half  the 
speed  of  the  loose  formation.  The  leader  executes  a  simple  adaption  system  which  selects  which 
formation  to  use  based  on  the  upcoming  hazards  and  the  remaining  mission  time  in  an  attempt 
to  minimize  danger,  while  maintaining  a  high  probability  of  mission  completion  by  time  T. 

The  DMPL  program  for  this  DRTS  consists  of  two  roles — leader  and  protector.  The  runtime 
instance  of  the  program  corresponding  to  the  system  consists  of  one  node  executing  the  leader 
role,  and  the  remaining  four  nodes  executing  the  protector  role.  The  leader  node  executes  three 
threads:  (i)  a  synchronous  thread  with  high  criticality  and  low-priority  that  implements  the 
collision  avoidance  protocol;  (ii)  an  asynchronous  thread  with  low  criticality  and  high  priority 
to  run  a  path  planning  algorithm  and  compute  the  next  waypoint;  and  (iii)  an  asynchronous 
thread  with  low  criticality  and  medium  priority  to  run  the  adaptation  algorithm  and  decide 
when  a  formation  change  is  needed.  The  adaptation  algorithm  itself  is  implemented  in  a  sepa¬ 
rate  component,  and  invoked  through  an  external  function  call.  Each  protector  node  runs  only 
two  threads — the  synchronous  collision  avoidance  and  the  asynchronous  path  planning.  On  each 
node,  the  path  planning  thread  communicates  with  the  collision  avoidance  thread  by  updating 
the  next  waypoint  coordinate.  In  addition,  the  adaptation  thread  on  the  leader  node  commu¬ 
nicates  with  the  waypoint  thread  (on  all  nodes)  by  updating  a  shared  variable  indicating  the 
formation  type.  Note  that  all  features  of  DMPL  are  required  to  implement  this  system. 
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Figure  3. 1 :  Example  DRTS  with  a  five-node  reconnaissance  mission 
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4  The  DMPL  Language 


A  DMPL  program  is  comprised  of  a  triple  ( GV,  LV,  R),  where  GV  is  a  set  of  global  variables,  LV 
is  a  set  of  local  variables,  and  R  is  a  set  of  roles.  A  role  is  comprised  of  a  pair  ( S ,  A)  where  S  is 
a  set  of  synchronous  threads,  and  A  is  a  set  of  asynchronous  threads.  A  thread  is  comprised  of  a 
triple  (tt,  k ,  /)  where  tt  is  its  period,  k  is  its  criticality,  and  /  is  its  job-function  that  is  executed 
in  each  period.  The  body  of  /  is  a  set  of  statements  that  accesses  local,  global,  temporary  and 
id  variables. 

DMPL  supports  calls  to  two  types  of  functions:  (i)  internal  functions  declared  as  part  of  the  DMPL 
program;  and  (ii)  external  C-| — b  functions.  External  functions  operate  on  an  external  state  and 
cannot  modify  dmpl  variables.  However,  they  can  return  values  that  can  be  used  by  their  caller 
internal  function  to  update  DMPL  variables.  The  job  function  of  a  thread  must  be  internal.  The 
set  of  all  internal  and  external  functions  are  denoted  IntFn  and  ExtFn,  respectively.  We  write 
Func  to  mean  the  set  of  all  functions,  i.e. ,  Func  =  IntFn  U  ExtFn. 


4.1  Abstract  Syntax 

Let  TV  be  a  set  of  temporary  variables,  IV  be  a  set  of  id  variables,  and  id  be  a  distinguished 
variable  such  that  GV,  LV,  TV,  IV  and  {id}  are  mutually  disjoint.  The  body  of  a  function  is 
a  statement.  The  abstract  syntax  of  statements,  lvalues  and  expressions  is  given  by  the  BNF 
grammar  in  Figure  4.1.  Intuitively,  skip  is  a  NOP,  l  =  e  is  an  assignment,  ite  is  an  if-tlien-else 
statement,  while  is  a  while  loop,  ALL(u,  st)  executes  st  iteratively  by  substituting  v  for  the  ID 
of  each  node,  (st i  ; . . . ;  stk)  executes  st i  through  stk  in  sequence,  v(v,  st)  introduces  a  fresh 
temporary  variable  v  in  scope  of  st,  ~  is  an  unary  operator,  and  o  is  a  binary  operator.  ALL 
enables  iteration  over  all  node  IDs  without  knowing  the  exact  number  of  such  IDs  a  priori. 
In  addition,  the  special  expression  0  denotes  the  void  value.  An  internal  function  /  with  k 
arguments  is  a  triple  (p,  b,  r)  where  p  £  TVk  is  the  list  of  its  arguments,  b  £  stmt  is  a  statement 
denoting  its  body,  and  r  £  exp  is  an  expression  denoting  its  return  value. 


4.1.1  Types  and  Operations 

All  dmpl  variables  are  typed.  These  types  and  operations  on  them  have  the  same  interpretation 
as  in  c-| — h  The  following  numeric  types  are  supported:  int  and  char  (both  signed  and 
unsigned  variants),  double,  bool.  Array  types  are  also  allowed.  Given  a  type  r  and  a  positive 
integer  n,  the  array  type  with  n  elements  of  type  r  is  denoted  by  r[n].  Only  numeric  and  bitwise 
operations  over  the  supported  types  are  allowed  (e.g.,  the  address  of  a  variable  cannot  be  taken). 
The  void  type  is  allowed  as  the  return  type  of  functions.  The  set  of  all  types  is  denoted  T.  The 
type  of  variable  v  is  denoted  typ(v). 


4.1.2  Scoping  and  Assumptions 

We  assume  that:  (i)  variables  in  GV  U  LV  U  {id}  are  always  in  scope;  (ii)  for  each  statement 
ALl(u,  st)  and  v(v,st),  variable  v  is  in  scope  of  st;  (iii)  scoping  is  unambiguous,  and  only 
variables  in  scope  are  used  in  expressions;  (iv)  id  and  id  variables  do  not  appear  on  the  LHS 
of  assignments,  i.e.,  they  are  read-only;  (v)  functions  are  called  with  arguments  of  proper  type, 
and  their  return  values  (if  any)  are  also  stored  into  variables  of  proper  type,  where  “proper” 
means  allowed  by  the  semantics  of  c-| — h  Note  that  these  assumptions  do  not  limit  expressivity. 
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(Statements)  stmt 


(LValues)  Ival 
(Expressions)  exp 


skip  |  Ival  =  exp 
ITE(exp,  stmt ,  stmt) 
WHILE  (exp,  stmt ) 

ALL(JV,  stmt) 

( stmt  ; . . . ;  stmt) 
Func(exp, . . . ,  exp) 

TV  =  Func(exp, . . . ,  exp) 
v{  TV ,  stmt) 

GV \LV \  TV 

Z  |  Ival  |  GF@/F 

id  |  /V  |~  exp  |  exp  o  exp 


Figure  4.1:  Grammar  of  statements  and  expressions 


4.2  Program  Instance 

An  instance  of  a  dmpl  program  P  of  size  n,  denoted  Pn,  is  a  finite  mapping  from  ids  in  the  range 
[0,  n)  to  roles.  This  induces  a  mapping  from  ids  to  threads  in  a  natural  manner,  if  Pn(i)  =  (S,  A ), 
then  S(i)  =  S  and  A(i)  =  A.  In  essence,  Pn  is  a  collection  of  n  nodes,  each  with  a  distinct  id  in 
the  range  [0,  n)  where  the  node  with  id  i,  denoted  W,  executes  the  role  Pn(i). 


4.2.1  Global  Variable  Expansion  and  Ownership 

Nodes  use  global  variables  to  communicate,  dmpl  enforces  every  global  variable  to  have  a  single 
writer  node,  as  follows.  In  the  program  instance  Pn ,  each  global  variable  v  of  type  r  is  expanded 
to  an  array  v  of  type  r[n].  Then,  node  iV,;  only  writes  to  the  element  v[i],  but  reads  from  all 
elements  of  v.  We  write  GV  to  mean  the  set  of  all  expanded  global  variables,  i.e. , 

GV  =  {F\ve  GV} 

As  we  shall  see  later,  this  restriction  will  be  useful  to  define  the  semantics  of  synchronous  threads. 


4.3  Semantics 


Semantically,  the  program  instance  Pn  is  a  distributed  system  with  n  communicating  nodes.  We 
define  this  semantics  formally  in  stages,  starting  with  functions  and  threads,  then  proceeding  to 
individual  nodes,  and  finally  defining  the  semantics  of  a  complete  program  instance.  The  set  of 
values  over  the  supported  types  T  is  denoted  D.  The  type  of  a  value  d  is  denoted  typ(d). 

4.3.1  Variable  Valuation 

Given  a  set  of  variables  V,  let  V(V)  be  the  set  of  partial  mappings  from  V  to  values  of  appropriate 
type.  In  other  words: 


Vs  €  V(V) .  \/v  €  Dom(s) .  typ(c)  =  typ(s(t>)) 
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Var(e)  =  < 


{ej 
{e} 
{vi,v2} 
Var(e') 
Var(e i)  U  Vdr(e2) 


if  e  G  GF 

if  eG  LVU  TV  U  IV  U  {id} 
if  e  =  G  GV@IV 
if  e  =~  e' 
if  e  =  e±  o  e2 

otherwise 


Figure  4.2:  Variables  appearing  syntactically  in  expressions 


e  >  s  =  < 


2  if  2  G  Z 
s(e)[i]  if  e  G  GV 
s(e)  if  e  G  LV  U  TV  U  IF  U  {id} 
s(ui)[s(u2)]  if  e  =  ui@U2  G  GV@IV 
~  (e'  >  s)  if  e  =~  e' 

[  (ei  >  s)  o  (e2  >  s)  if  e  =  ei  o  e2 


Figure  4.3:  Evaluation  of  expressions  for  node  N, 


For  any  non-void  type  r,  the  initial  value  J(r)  is  0  cast  to  r  as  per  the  semantics  of  C-| — h  Let 
1(F)  G  V(F)  be  the  special  initial  mapping  that  maps  each  variable  v  G  V  of  type  r  to  7(r). 
We  use  the  terms  state  and  valuation  interchangeably. 

Let  Var{e )  denote  the  set  of  variables  appearing  syntactically  in  expression  e,  as  defined  in 
Figure  4.2.  Let  e  be  an  expression  and  s  be  a  state  such  that  Var(e)  C  Dom(s).  The  evaluation 
of  e  under  s,  denoted  e  >  s,  is  defined  inductively  over  the  structure  of  e  in  a  natural  manner, 
as  shown  in  Figure  4.3.  Note  that  operators  are  interpreted  as  in  CH — h,  and  global  variables 
evaluate  to  the  *-th  element  of  their  expanded  version.  Also  note  that  ec>s  is  well-defined  under 
our  assumption  Far(e)  C  Dom(s)  since  we  always  apply  the  mapping  s  to  a  variable  in  its 
domain.  We  write  s  |=  e  to  mean  that  the  value  el >  s  would  be  converted  to  true  under  the 
Boolean  conversion  rules  of  C++-  The  void  expression  ©  evaluates  to  itself  under  all  states, 
i.e.,  ©  >  s  =  ©.  If  an  array  is  read  out  of  bounds,  the  result  is  0  cast  to  the  type  of  the  array 
element.  If  an  array  is  written  out  of  bounds,  it  is  a  NOP. 

We  define  three  operations  on  states:  (i)  Update:  given  a  state  s,  variable  v.  and  value  d,  the 
updated  state  s[u  >->•  d]  is  identical  to  s  except  that  it  maps  v  to  d;  (ii)  Extension:  given  a  state 
s  and  a  variable  v  $  Dom(s)  of  type  r,  s  ©  v  is  that  state  such  that: 

Dom(s  ©  v)  =  Dom(s)  U  {z;} 

W  G  Dom{s) .  (s  ©  v)(v')  =  s(v') 

( s  ©  v)(v)  =  I(t ) 

and  (iii)  Restriction:  given  a  state  s  and  a  variable  v  G  Dom{s),  s  ©  v  is  that  state  such  that: 

Dom(s  0  v)  =  Dom(s)  \  {u} 

W  G  Dom{s  0  v) .  (s  0  v)(v')  =  s(v') 

Thus,  s  ©  v  extends  s  with  a  new  variable  v  set  to  its  initial  value,  while  s  Qv  projects  away  v. 
We  write  s  ©  (v,  d)  to  mean  the  state  (s  ©  v)[u  >->•  d] ,  i.e.,  the  state  obtained  from  s  by  adding 
a  new  variable  v  initialized  to  value  d.  We  use  state  update  to  handle  assignments,  and  state 
extension  and  restriction  to  handle  temporary  variables.  Given  a  state  s  and  a  set  of  variables 
V,  we  obtain  s\  V  by  removing  from  s  mappings  for  variables  in  V  and  s(~lF  by  removing  from  s 
mappings  for  variables  not  in  V .  Given  two  states  Si  and  S2  such  that  Dom{si)  fl  Dom(s2)  =  0, 
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the  state  si  ©  s-2  denotes  their  union: 


Dom{si  ©  S2)  =  Dom(si)  U  Dom(s2) 

V*  €  [1, 2]  .  Vv  £  Dom(si) .  (si  ©  s2)(v)  =  s,;(u) 


4.3.2  Thread  Jobs 

Threads  in  DMPL  execute  periodically  in  a  read-execute-write  manner.  Semantically,  a  thread 
t  =  (7 r,  k,  f)  consists  of  an  infinite  sequence  of  jobs  that  activate  at  times  0, 7r,  27t,  . . .  etc.  Each 
job  atomically  copies  local  and  global  variables  at  the  beginning  into  temporary  variables,  then 
operates  over  these  temporary  variables  to  update  their  values  by  executing  function  /,  and 
finally  atomically  writes  the  new  values  back  to  the  corresponding  local  and  global  variables. 
Note  that  due  to  thread  scheduling,  a  job  may  not  start  execution  immediately  on  activation. 
It  may  also  be  preempted  by  other  jobs  during  execution. 


4.3.3  Thread  Scheduling 

All  DMPL  threads  are  real-time — that  is,  they  have  worst  case  execution  times  (WCETs)  and 
deadlines.  We  assume  that  threads  which  belong  to  a  node  execute  on  a  single  CPU.  Threads 
are  scheduled  based  on  their  priorities  (7 r)  and  criticalities  (k)  using  Zero-Slack  Rate  Mono¬ 
tonic  (zsrm)  scheduling  [9].  At  a  high  level,  this  means  that  threads  with  higher  priority  are 
preemptively  allocated  the  CPU  during  normal  operating  conditions,  while  threads  with  higher 
criticality  are  preemptively  allocated  the  CPU  during  overload  operating  conditions.  We  assume 
that  threads  are  schedulable — that  is,  a  job  that  starts  at  time  i  x  n  completes  execution  before 
time  (*  +  1)  x  7r  under  the  worst  case  interference  from  the  other  threads.  This  is  achieved  by 
using  an  offline  ZSRM  schedulability  analysis  [9].  WCET  estimation  is  a  challenging  problem  in 
its  own  right  [32],  especially  in  the  presence  of  caches  and  other  components.  As  is  common  in 
schedulability,  we  assume  that  WCET  estimation  is  handled  through  a  separate  procedure  that 
covers  both  code  generated  from  dmpl  and  external  libraries. 


4.3.4  External  Function  Semantics 

For  each  external  function  /  with  k  parameters,  we  assume  a  semantics  [/]  C  Dk  x  D.  Formally, 
(di, . . . ,  dk,  d)  £  [/]  if  and  only  if  the  invocation  of  /  with  arguments  d±, . . . ,  dk  could  return 
with  value  d. 

4.3.5  Statement  Semantics 

Let  V  =  GV  U  LV  U  TV  U  IV  U  {id}  be  the  set  of  all  variables,  and  Vs  =  GV  U  LV  be  the 
set  of  shared  (i.e. ,  global  and  local)  variables.  The  semantics  of  a  statement  st  is  given  by  the 
way  it  updates  the  values  of  variables,  and  the  shared  variables  to  which  it  writes.  Formally,  it 
is  the  relation  [st]  C  V(V)  x  V(V)  x  2Vs  defined  inductively  over  structure  of  st.  Let  us  write 
{s}[st\W,w}  to  mean  ( s,s',w )  £  [st].  Then,  [•]  is  the  smallest  relation  satisfying  the  rules  in 
Figure  4.4.  A  high-level  description  of  the  proof-rules  is  as  follows: 

•  SKIP:  The  skip  statement  has  no  effect  on  the  state. 

•  ASGN-GLOBAL:  Assignment  to  a  global  variable  v  is  semantically  an  assignment  to  the  i-th 
element  of  its  expanded  version  v. 
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•  ASGN-LOC  and  ASGN-TMP:  Assignments  to  local  and  temporary  variables  update  the  lhs 
based  on  the  evaluation  of  the  rhs  in  the  current  state. 

•  ite-IF  and  ITE-THEN:  If-then-else  statements  execute  the  appropriate  branch  based  on  the 
evaluation  of  their  condition  in  the  current  state. 

•  while-exit  and  WHILE-LOOP:  A  while  loop  either  terminates  immediately  if  the  loop 
condition  fails,  or  executes  the  body  once  and  repeats. 

•  ALL:  The  iteration  statement  executes  the  body  repeatedly,  assigning  values  [0,  n)  to  the 
fresh  id  variable. 

•  SEQ:  A  sequence  of  statements  operates  as  expected,  updating  the  state  by  executing  its 
components  one  after  the  other. 

•  EXT-FUNCl  and  EXT-FUNC2:  External  functions  optionally  return  a  value  which  is  stored 
in  a  temporary  variable. 

•  iNT-FUNCl  and  INT-FUNC2:  Internal  functions  update  local  and  global  variables  only. 
They  also  optionally  return  a  value  which  is  stored  in  a  temporary  variable. 

•  NEW-VAR:  Fresh  variables  exist  only  within  the  scope  of  their  definition. 

•  function:  The  arguments  of  a  function  determine  its  semantics.  From  the  perspective  of 
its  caller,  a  function  updates  local  and  global  variables  and  returns  a  value. 


4.3.6  Thread  Semantics 

Each  thread  t  =  (n,K,  f)  G  SUd  executes  an  infinite  sequence  of  jobs  jo,ji,  ■  ■  ■ ,  with  each 
job  having  an  arrival  time  (when  it  becomes  active)  and  a  departure  time  (when  it  completes 
execution  of  its  function  or  aborts).  In  addition,  each  job  reads  all  shared  variables  before 
executing  /\jmd  writes  the  updated  values  of  shared  variables  back  after  executing  /.  Recall 
that  Vs  =  GV  U  LV  is  the  set  of  shared  (global  and  local)  variables.  We  denote  the  arrival  of 
a  job  with  the  symbol  f,  the  departure  of  a  job  with  the  symbol  l,  and  the  abort  of  a  job  with 
the  symbol  f.  Thus,  an  execution  of  t  is  an  infinite  sequence: 

(  (O5  T?  Co,  so,  0),  (0,  eo,  Cq,  Sq,  icq), 

(1, t, Cl, Si, 0),  (l,ei,ci,si,tyi),  . ..) 

such  that  the  following  conditions  hold: 

•  Events,  timestamps,  and  states  have  the  correct  type: 

Vi  >  0  .  e.j  G  {!,  f}  A  Ci,  c'elA  Sj,  s'  G  V(VS)  A  w[  C  Vs 

•  Jobs  arrive  periodically,  require  positive  time  to  execute,  and  timestamps  are  non¬ 
decreasing: 

Vi  >  0  .  Cj  G  [j  x  7 r,  (i  +  1)  x  7r)  A  c,;  <  c'  A  c'  <  (i  +  1)  x  7r 

•  The  state  is  properly  initialized:  so  =X(YS). 

•  If  a  job  aborts,  it  leaves  the  state  unchanged.  Otherwise,  it  updates  the  state  according 
to  the  semantics  of  its  function: 

Vi  >  0.  (a  =1  =>  j>i}[/]{s',u>'})  A 
(e*  =  t  =>  {s'i  =  Si  A  w'  =  0)) 
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Intuitively,  job  ji  starts  executing  at  time  Cj,  reads  state  Sj,  and  either  updates  it  to  s'  by 
executing  function  /,  or  aborts  and  leaves  the  state  unchanged,  and  finally  departs  at  time 
c'.  Moreover,  the  first  job  reads  the  initial  values  of  all  variables.  The  semantics  of  thread  t, 
denoted  [f],  is  the  set  of  all  its  executions. 


4.3.7  Role  Semantics 

Next,  we  combine  the  executions  of  all  threads  into  an  execution  of  the  role.  In  doing  so,  we 
must  ensure  that  threads  are  scheduled  correctly  based  on  their  priorities,  and  that  states  are 
observed  and  updated  in  a  sequentially  consistent  manner,  i.e. ,  when  a  job  arrives,  its  reads  the 
most  recently  written  values  of  local  variables.  Note  that  we  do  not  restrict  values  of  global 
variables  since  these  are  shared  across  nodes  and  hence  their  values  are  determined  by  the 
inter-node  communication.  Consider  role  p  =  (S,  A).  An  execution  of  p  is  an  infinite  sequence: 

V  —  ((du  e0i  co>  so>  wo>  £o)>  ('i>  ei>  cii  si,  . . .) 

such  that  the  following  conditions  hold: 

•  Each  f  j  £  S  U  A  is  a  thread. 

•  For  each  thread  t  £  SU  A,  the  projection  of  p  on  t ,  i.e.,  the  subsequence  of  p  consisting  of 
elements  with  ti  =  t,  is  an  execution  of  t. 

•  Timestamps  are  non-decreasing,  i.e.,  VI  >  0  .  Cj  <  Cj+i. 

•  A  lower  priority  thread  ti  can  only  preempt  a  higher  priority  thread  th  when  it  is  in 
overload  and  has  higher  criticality  than  th-  In  ZSRM,  a  thread  is  said  to  be  in  overload  if 
it  has  an  active  job  and  the  current  time  is  no  less  than  the  job’s  zero-slack  instant  (ZSI). 
Informally,  the  ZSI  of  a  job  is  the  latest  time  at  which  it  can  be  started  (or  resumed)  so 
that  it  completes  its  remaining  execution  before  its  deadline.  Given  a  thread  t  and  a  job 
index  i ,  we  write  ZSI(t-,i)  to  denote  the  ZSI  of  the  (,-th  job  of  t.  We  note  that  ZSI(t,L ) 
is  statically  computable  via  the  ZSRM  schedulability  analysis.  Thus,  whenever  p  has  a 
sub-sequence: 

(('li  ei,  Cl,  Si,  tl),  ('2j  e2,  C2,  S2,  tv),  ('3,  e3,  C3,  S3,  £3)) 

such  that: 

tl  =  t3  A  '1  =  (-3  A  7T (ti)  >  7T (f2) 
then  the  following  must  be  true: 

«(t2)  >  ft(ti)  A  c2  >  ZSI(ti,ii) 

•  Each  arriving  job  reads  the  value  of  local  variables  written  by  the  most  recently  success¬ 
fully  completed  job.  For  each  i  >  0,  and  v  €  LV,  let  PrWr(i,v)  be  the  set  of  indices 
corresponding  to  previous  successfully  completed  jobs  that  wrote  to  v ,  i.e., 

PrWr(i,  v)  =  {j  <  i  \  ej  =4.  Ai>  £  Wj] 


Then: 


VI  >  0 .  e,  =t  =>  Vu  £  LV  .  Si{v)  =  sm(v), 
where  m  =  max({0}  U  PrWr(i )) 

Note  that  in  the  absence  of  prior  writes  to  v,  m  evaluates  to  0,  and  the  initial  value  of  v 
is  read. 
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4.3.8  Program  Instance  Semantics 


Finally,  we  obtain  the  semantics  of  a  program  instance  by  combining  the  semantics  of  all  the 
roles.  In  doing  so,  we  must  ensure  that  the  values  of  global  variables  are  propagated  across 
nodes  in  a  consistent  manner.  Since  dmpl  nodes  communicate  over  unreliable  networks  with 
no  synchronized  clocks  and  no  guaranteed  order  of  message  delivery,  we  do  not  have  any  total 
sequential  consistency  between  nodes  in  terms  of  how  global  variables  are  modified  and  observed. 
Informally,  when  a  job  reads  a  global  variable  v,  it  must  observe  a  value  no  earlier  than  the  last 
time  it  read  v.  However,  if  the  reader  and  writer  nodes  are  the  same,  the  most  recent  write  is 
observed  since  no  network  messages  are  involved  in  this  case.  Figure  4.5  shows  two  executions  to 
highlight  this  situation.  Formally,  consider  a  program  instance  Pn.  Recall  that  Pn  is  a  mapping 
from  [0,  n)  to  roles.  An  execution  of  Pn  is  an  infinite  sequence 


V  —  ((h),  e0,  c0,  s0,  wo,  Wo),  (H,ei,ci,si,  wi,t\,pi), . . .) 

such  that  the  following  conditions  hold: 

•  Each  pi  is  a  role. 

•  For  each  role  p  £  R,  the  projection  of  77  on  p.  i.e. ,  the  subsequence  of  77  consisting  of 
elements  with  p,  =  p,  is  an  execution  of  p. 

•  Each  arriving  job  reads  the  value  of  a  global  variable  that  is  no  older  than  the  last  value 
that  was  read.  Formally,  this  means  that  for  each  global  variable  v  £  GV,  each  writer 
node  Nw,w  £  [0,  n),  and  each  reader  node  Nr,r  £  [0,  n),  there  is  a  monotonic  mapping 
<&(?;,  w,  r)  :  N  1— >  N  from  each  position  where  a  value  of  v\w\  is  read  by  Nr  to  the  position 
where  the  corresponding  value  was  written  by  Nw.  Thus: 

Vi  £  N .  pi  =  Pn(r)  A  Ci  =f  => 

p<s>(i)  =  Pn(w)  A  e$(i)  =4  As, (v) [w]  =  s$(i)(u)[w] 

The  mapping  is  monotonic,  thus  older  writes  are  never  observed  after  newer  ones: 

Vi  <i' .  pi  =  pi’  =  Pn (r)  A  e*  =  ei>  =f  =>  <f>(i)  <  <f>(i') 

Note  that  the  mapping  w,  r)  varies  with  v,  w  and  r.  Thus,  due  to  network  delays, 
writes  to  the  same  variable  could  be  observed  in  different  order  by  different  readers,  and 
writes  to  different  variables  could  be  observed  by  the  same  reader  in  different  order.  How¬ 
ever,  if  the  reader  and  writer  nodes  are  the  same,  then  there  are  no  network  messages 
involved,  and  the  most  recent  write  is  observed  (as  for  local  variables).  Formally,  for 
each  i  >  0,  global  variable  v  £  GV,  and  node  Nr,  let  PrWr(i,v,r)  be  the  set  of  indices 
corresponding  to  previous  successfully  completed  jobs  of  Nr  that  wrote  to  v[r\,  i.e., 

PrWr(i,v,r)  =  {j  <  i  \  ej  =4.  An[r]  £  Wj} 


Then: 


Vi  £  N .  pi  =  Pn  (r )  A  ej  =t  => 

4>(u,  r,  r)(i)  =  max({0}  U  PrWr(i,v,  r)) 


Note  that  in  the  absence  of  prior  writes  to  v[r],  4>(u,  r,  r)(i)  evaluates  to  0,  and  the  initial 
value  of  v\r\  is  observed. 


Each  synchronous  thread  t  executes  in  rounds.  Let  S(t)  be  the  ids  of  the  nodes  that 
execute  t ,  i.e., 

S(t)  =  {j  \Pn(j)  =  (S,A)At£S} 
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For  each  round  i  >  0,  and  node  id  j  £  S(t),  let  Round+  (t,i,  j)  be  the  index  in  ??  corre¬ 
sponding  to  the  start  of  the  i-th  job  of  thread  t  in  node  Nj  that  completes  successfully, 
and  Round,- (t,i,  j)  be  the  index  in  77  corresponding  to  the  end  of  the  i-th  job  of  thread  t 
in  node  Nj  that  completes  successfully.  Then: 

\/j'  £  S(t) .  Round~  (t,i,  j')  <  Round+  (t,i  +  1  ,j) 

Moreover,  the  value  of  a  global  variable  read  at  the  beginning  of  round  i  +  1  equals  the 
value  written  at  the  end  of  round  i.  Recall  the  definition  of  w,  r).  Thus: 

$(u,  w,  r)(Round+  (t,  i  +  1,  r))  =  Round~  (t,  i,  w) 


4.4  Property  Specification 

As  mentioned,  we  have  explored  verification  of  properties  implemented  by  synchronous  threads. 
Such  properties  are  expressed  as  round  invariants:  conditions  that  must  hold  at  the  beginning 
of  each  round  of  execution  of  a  synchronous  thread.  Formally,  a  specification  is  a  pair  ( t ,  /) 
where  t  is  a  synchronous  thread,  and  /  is  an  internal  function  that  evaluates  the  round  invariant, 
and  returns  it  as  a  bool.  Internal  functions  used  in  properties  (a.k.a.  property  functions)  are 
distinguished  from  other  internal  functions  in  two  ways: 

1.  They  do  not  modify  any  shared  variables. 

2.  They  can  refer  to  local  variables  of  arbitrary  nodes.  This  is  necessary  because  a  property 
function  has  to  evaluate  relations  over  the  states  of  multiple  nodes.  For  instance,  in  the 
case  of  collision  avoidance  in  our  example,  the  property  function  checks  that  every  pair  of 
distinct  nodes  occupies  different  cells  (i.e.,  each  pair  has  different  values  of  x  or  y). 

Property  functions  are  evaluated  over  program  instance  states.  A  program  instance  state  is  a 
pair  ( si,sg )  such  that  s;  :  [0,n)  >->•  V(LV)  and  sg  £  V(GV).  Informally,  si  maps  each  node  id  i 
to  the  value  of  local  variables  at  Ni,  while  sg  is  a  mapping  from  global  variables  to  their  values. 
In  general,  since  Pn  is  distributed,  it  may  not  have  a  well-defined  program  instance  state  at 
all  times  during  an  execution.  In  particular,  the  value  of  a  global  variable  may  be  different  at 
different  nodes.  However,  at  the  beginning  of  each  round  of  a  synchronous  thread,  there  is  a 
unique  program  instance  state  since  all  nodes  must  agree  on  the  value  of  each  global  variable. 
We  denote  by  S(tt,  t,  i)  the  program  instance  state  at  the  beginning  of  round  i  of  t  in  execution 
77- 

Given  a  program  instance  state  s  =  (s;,  sg),  and  a  property  function  /,  we  write  /(s)  to  denote 
the  value  returned  by  /  when  invoked  from  s.  This  is  defined  similarly  to  a  normal  internal 
function,  except  that  s;[i]  is  used  to  lookup  the  value  of  local  variables  for  Ni,  while  sg  is  used 
to  lookup  the  value  of  global  variables.  Finally,  we  say  that  a  program  instance  Pn  satisfies  a 
specification  (t,  /),  denoted  Pn  |=  (t,  /),  if  for  every  execution  77  of  Pni  we  have: 

v*>o./(£fa,M))  =  f 


4.5  Concrete  Syntax 

The  concrete  syntax  of  DMPL  consists  of  declarations  for  GV  and  LV  plus  definitions  of  func¬ 
tions,  threads,  roles,  and  properties.  For  example,  Figure  4.6  shows  a  fragment  of  the  DMPL 
program  corresponding  to  our  example  described  in  Chapter  3.  Note  the  following: 
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•  The  syntax  is  similar  to  CH — h  This  provides  familiarity  to  practitioners  and  simplifies 
code  generation  and  sequentialization. 

•  Platform-specific  code  is  included  at  the  top  inside  7.7.  {  .  .  .  7.7.}  blocks.  This  feature  is 
used  to  include  code  (header  files,  helper  functions,  etc.)  needed  to  generate  compilable 
C++.  This  code  is  not  subjected  to  any  analysis.  It  is  emitted  verbatim  during  code 
generation.  The  programmer  should  ensure  that  this  code  does  not  conflict  with  the  code 
generated  from  the  rest  of  the  DMPL  program  (e.g.,  no  variable  name  clashes)  or  cause 
other  compilation  problems  (e.g.,  syntax  errors).  This  code  is  ignored  during  verification. 
In  Figure  4.6,  this  code  is  used  to  initialize  the  hazard  levels  in  each  grid  cell  randomly, 
and  to  periodically  access  hazard  levels  and  compute  the  total  exposure  faced  by  the  leader 
at  runtime. 

•  Constant  definitions  (lines  14-15)  are  allowed  for  readability. 

•  Multi-dimensional  array  variables  are  supported. 

•  External  functions  are  declared  using  the  extern  keyword.  External  functions  can  also 
be  labeled  pure  (e.g.,  line  19).  This  is  a  hint  that  the  function  should  not  modify  the 
external  system  state.  It  is  not  checked  or  enforced  in  any  way.  In  our  case,  we  expect 
that  the  GET_HAZARD  ( )  function  should  be  read-only.  External  functions  are  also  declared 
or  defined  in  C-| — |-  within  the  code  included  via  the  7.7.  { .  .  .  7.7.}  block. 

•  Shared  variables  can  be  initialized  to  non-zero  values.  This  is  done  either  via  direct 
assignment  (line  26),  or  via  constructors  (line  34). 

•  Shared  variables  can  also  be  labeled  as  input  and  constraints  can  be  imposed  on  their 
initial  values  (lines  28-29).  For  example,  variable  x  is  constrained  to  be  between  0  and 
X-l  initially.  Input  variables  are  treated  dually  as  follows:  (i)  the  generated  code  allows 
their  initial  values  to  be  specified  from  the  command  line,  but  performs  a  runtime  check 
that  any  such  value  respects  specified  constraints;  and  (ii)  during  verification,  their  initial 
values  are  assumed  to  be  non-deterministic  subject  to  the  constraints.  This  ensures  that 
our  verification  is  sound. 

•  Threads  are  defined  with  the  thread  keyword.  The  ©BarrierSync  annotation  indicates 
a  synchronous  thread.  Other  annotations  (©Period  for  period,  ©Criticality  for  criti¬ 
cality,  ©WCExecTimeNominal  for  WCET  under  normal  execution,  ©WCExecTimeOverload 
for  WCET  under  overload)  are  used  to  specify  parameters  used  for  thread  schedulability 
analysis  and  scheduling.  Time  parameters  are  specified  in  microseconds. 

•  Iterators  are  available  to:  (i)  execute  a  statement  over  all  nodes  (f  orall_node),  all  pairs 
of  distinct  nodes  (f  orall_distinct_node_pair),  etc.;  and  (ii)  evaluate  an  expression  dis¬ 
junctively  over  nodes  that  have  a  lower  id  (exists_lower),  a  higher  id  (exists_higher), 
etc.  They  are  all  “syntactic  sugar”  defined  formally  using  all  in  a  natural  manner. 

•  A  role  can  use  threads  defined  in  the  global  scope.  For  example,  the  C0LLISI0N_AV0IDANCE 
thread  is  used  by  both  the  Leader  (line  71)  and  Protector  (line  90)  roles.  A  role  can 
redefine  timing  parameters  of  imported  threads.  For  example,  the  Leader  roles  redefines 
the  WCET  estimates  (lines  73-74)  of  the  WAYPOINT  thread.  A  role  can  also  define  new 
threads.  For  example,  the  ADAPTATION_MANAGER  thread  is  defined  by  the  LEADER  roles, 
since  the  functionality  implemented  by  this  thread  is  required  only  by  the  leader  node. 

•  A  role  can  redefine  (or  override)  the  initialization  of  shared  variables.  For  example,  the 
Leader  role  redefines  variables  xt  and  yt  to  be  inputs.  This  is  necessary  since  the  final 
target  cell  of  the  leader  is  part  of  the  mission  parameters  and  is  supplied  as  as  input  to 
the  system.  Note  that  these  variables  are  initialized  to  the  default  value  0  (line  30)  for  the 
Protector  role.  This  makes  sense  since  the  target  cell  for  a  protector  changes  over  time 
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as  the  leader  moves  because  each  protector  tries  to  move  to  a  specific  position  relative  to 
the  leader  in  order  to  maintain  the  overall  formation  (cf.  Figure  3.1). 

•  A  role  can  override  the  definition  of  an  internal  function,  and  invoke  the  base  function 
if  needed.  For  example,  the  Leader  role  overrides  the  REACHED_NEXT_XY  function  (line  66) 
since  the  leader  must  execute  additional  code  (compared  to  a  protector)  once  it  reaches  the 
next  waypoint.  It  invoked  the  base  definition  of  REACHED_NEXT_XY  (line  68)  to  implement 
functionality  common  with  the  protectors. 

•  Internal  functions  can  be  labeled  pure  (e.g.,  NoCollisions  at  line  98).  Such  functions 
may  not  modify  shared  variables.  This  is  checked  by  the  compiler,  and  compilation  aborts 
if  a  violation  is  detected.  Since  there  is  no  aliasing  in  DMPL,  this  check  simply  ensures  that 
no  shared  variables  are  on  the  lhs  of  assignments  in  the  body  of  the  function,  and  that 
all  internal  functions  invoked  by  this  function  are  also  labeled  pure.  Recall  that  external 
functions  cannot  modify  shared  variables  indirectly. 

•  Properties  are  defined  via  the  require  keyword  (line  107).  The  property  function  (in  this 
case  NoCollisions)  must  be  pure  to  ensure  that  it  has  no  side  effects  on  the  program 
state.  Note  that  in  our  example,  the  property  function  does  indeed  refer  to  local  variables 
x  and  y  of  multiple  nodes. 

Since  dmpl  is  an  experimental  language,  the  concrete  syntax  presented  here  is  not  final.  How¬ 
ever,  the  concepts  presented  above  should  persist  since  they  emerged  from  the  need  to  program 
a  realistic  system,  and  we  expect  them  to  be  useful  for  programming  DRTS  in  general. 
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[SKIP] 


{s}[sfczpj{s,0} 

v  G  LV  w  =  {zj} 
{s}[z;  =  e]{s[f  i  ^  e  >  s],  zu} 

s\=e  |s}[sti]{s,,zz)} 
{s}[lTE(e,  sti,  s£2)]{s',  w} 


[ASGN-LOC] 

[ITE-THEN] 

-‘{s  t=  e) 


v  G  GV  w  =  {z’[z]} 
{s}{v  =  e]{s[z;[z]  i  y  e  i>  s],  w} 

v  G  TV 


{s}[z;  =  e]{s[z>  >-»•  e  >  s],  0} 

-n(s|=e)  {3}|at2]{a>} 

{s}[lTE(e,  sti,  s£2)]{s',  w} 

[WHILE-EXIT] 


[ASGN-GLOBAL] 

[ASGN-TMP] 

[ITE-ELSE] 


{s}[WHILE(e,s£)]{s,0} 

s\=e  {s} [[si] {s',  ttz'}  {s'}[WHlLE(e,  s£)]{s",  w"}  w  =  w'Uw' 
{s}[WHILE(e,  5i)]{s,,1  w} 


[WHILE-LOOP] 


v  ^  Dom(s)  so  =  s  ©  v 

{s0[z>  H>  0]}[si]{si, zz>i}  {si[v  H>  l]}fsf]{s2,zti2}  . . .  {s„_i[z;  H>  n  -  l]}[s£]{sn,  wn} 
s'  =  sn  Q  v  w  =  zz>i  U  •  •  •  U  wn 


{s}(ALL(v,  st)j{s' ,w} 

{s0}[s£i]{si,Zt>i}  .  .  .  {Sfc_i}[s£fc]{sfe,  Wk}  w  =  Wi  U  •  •  •  U  Wk 
{s0}I(s£i;...;s£fe)]{sfc,zz;} 

/  G  ExtFn  (ei  >  s, . . . ,  ek  >  s,  d)  G  [/] 
M[/(ei,...,efc)]]{s,0} 

/  G  ExtFn  {ei>  s, . . .  ,ek>  s,d)  G  [/]  s'  =  s[zt  i— >  d] 


[ALL] 


[SEQ] 


Mb  =  /(ei,...,efc)]{s,,0} 


[EXT-FUNCl] 

[EXT-FUNC2] 


/  G  IntFn  d\  =  ei>  s, . . .  ,dk  =  ek>  s 
si  =  s  \  TV  {si}[/(di,, . .  ,dk)]{s2,d,w}  s'  =  s2  ®  (s  n  TV) 
{s}[f(ei,...,ek)j{s',w} 


[INT-FUNC1] 


/  G  IntFn  d\  =  e\  t>  s, . . . ,  dk  =  ek  t>  s 
si  =  s  \  TV  {si}[/(di,...,dfc)]{s2,d,zy}  s'  =  s2  ®  (s  D  TV)[v  i-f  d] 


{s}fu  =  f{ei, . . . , ek)j{s' ,w} 

v  £  Dom(s)  s\  =  s(Bv  {si}[s£]{s2,  zz>}  s'  =  s2  0  v 
{s}[^(zz,  si)]{s',  tz;} 


[INT-FUNC2] 


[NEW-VAR] 


/  =  (p,b,r)  G  IntFn  s  G  V{GV  U  LV)  p  =  (v i,...,vk) 

Si  =  s  ®  (fi,di)  ®  . . .  ©  (vk,  dfc)  {si}[6]{s2,zy}  s'  =  (s2  0  Vi  0  . . .  0  vk)  d  =  r  >  s2 

{s}lf(d1,...,dk)]{s',d,w} 


[FUNCTION] 


Figure  4.4:  Execution  semantics  of  statements  and  functions  for  node  N,  in  program  instance 

Pn 
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N0  N0 


»  . > 


Figure  4.5:  Two  executions  of  a  program  instance  P3.  Time  flows  from  top  to  bottom;  r(v[i\,  d) 
=  value  d  was  read  from  v[i\;  u>(v[i\,d)  =  value  d  was  written  to  v[i\.  The  execution  on  the  left 
is  legal  even  though  N2  reads  a  stale  value  d\,  since  it  observed  no  previous  writes  to  v.  The 
execution  on  the  right  is  illegal  for  two  reasons:  (i)  once  Ni  reads  value  d2,  it  can  no  longer 
read  the  older  value  dx;  and  (ii)  N0  must  always  read  the  most  recent  write  to  v. 
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#include  <map> 

#include  "adaptation_manager .h" 

double  accumulated_risk  =  0.0; 

map  <  int  ,  map  <  int  ,  double  >  >  hazard_map; 

void  INIT_HAZARDS (int  _X , int  _Y , 

int  tx  ,  int  ty)  {  .  .  .  } 
double  GET_HAZARD  C  int  x  ,  int  y)  {  .  .  .  } 
double  updat  e_r  i sk  (  double  risk)  {  .  .  .  } 

•///.} 

/***  constants  ***/ 

const  X  =  20;  const  Y  =  20; 

const  INITS  =  0;  const  NEXT  =  1;  ... 

/***  external  function  declarations  */ 
extern  void  INIT_HAZARDS (. . .) ; 
extern  pure  double  GET_HAZARD  (  .  .  .  )  ; 
extern  double  update_risk  (...); 
extern  bool  adapt  at ion_manager  (...); 
extern  int  GRID_M0VE C uns igned  char  _xp  , 
unsigned  char  _yp  ,  double  _z); 

/*  shared  variables  &  initialization  */ 
local  unsigned  char  state  =  INITS; 

//current,  next  and  target  coordinates 
local  unsigned  char  input  x~(0<=x  &&  x<X); 
local  unsigned  char  input  y"(0<=y  &&  y<Y); 
local  unsigned  char  xp  =  x,  yp  =  y; 
local  unsigned  char  xt ,  yt ; 

//current  formation 

global  bool  input  formation; 

global  bool  lock  [X]  [Y]  =  {  lock  [x]  [y]  =  1;  }; 

/***  internal  functions  ***/ 

//--  compute  the  next  grid  on  path 
bool  NEXT.XY  ()  {  ...  > 

//--  stuff  once  next  waypoint  reached 
void  REACHED_NEXT_XY ()  {  ...  > 

/***  threads  ***/ 

©BarrierSync ; 

©Period (100000) ;  ©Criticality (4) ; 
©WCExecTimeNominal C15000) ; 

©WCExecTimeOverload  (30000)  ; 
thread  C0LLISI0N_AV0IDANCE  { 

if (state  ==  INITS)  {  ...  state  =  NEXT;  > 
if (state  ==  NEXT)  { 

if ( NEXT_XY ( ) )  return;  state  =  REQUEST; 

>  .  .  . 

else  if (state  ==  MOVE)  { 

if ( GRID_M0VE ( xp , yp , 0 . 5) )  return; 
REACHED_NEXT_XY () ;  ...  state  =  NEXT; 

> 

> 


57  ©Period (100000)  ; 

58  ©Criticality (3)  ; 

59  thread  WAYPOINT; 

60 

61  /***  roles  ***/ 

62  role  Leader  { 

63  override  local  unsigned  char  input  xt , 

64  input  yt ; 

65 

66  override  void  REACHED_NEXT_XY () 

67  { 

68  base . REACHED_NEXT_XY () ;  ... 

69  > 

70 

71  thread  C0LLISI0N_AVQIDANCE ; 

72 

73  ©WCExecTimeNominal ( 10000) ; 

74  ©WCExecTimeOverload (20000) ; 

75  thread  WAYPOINT  {  ...  > 

76 

77  ©Period (4000000)  ; 

78  ©Criticality (2)  ; 

79  ©WCExecTimeNominal ( 10000) ; 

80  ©WCExecTimeOverload (20000)  ; 

81  thread  ADAPTATI0N_MANAGER 

82  { 

83 

84  formation  =  adapt  at ion_manager (...) ; 

85 

86  > 

87  > 

88 

89  role  Protector  -( 

90  thread  C0LLISI0N_AV0IDANCE ; 

91 

92  ©WCExecTimeNominal (10000)  ; 

93  ©WCExecTimeOverload (20000)  ; 

94  thread  WAYPOINT  {  ...  > 

95  > 

96 

97  /***  properties  ***/ 

98  pure  bool  NoCollisions  () 

99  { 

100  f  orall_distinct_node_pair  (idl,id2)  -( 

101  if  (x©idl  ==  x©id2  &&  y@idl  ==  y©id2) 

102  return  false; 

103  > 

104  return  true  ; 

105  > 

106 

107  require  C0LLISI0N_AV0IDANCE  =>  NoCollisions; 


Figure  4.6:  dmpl  program  for  5-robot  reconnaissance  example 
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5  Code  Generation 


In  this  section,  we  describe  how  CH — R  code  can  be  generated  from  a  DMPL  program,  given  a 
mapping  from  node  ids  to  roles.  Figure  5.1  shows  a  fragment  of  the  CH — R  code  generated  from 
the  DMPL  program  shown  in  Figure  4.6.  Notable  features  of  this  CH — b  code  are: 

•  Code  within  the  %%-C .  .  .  7.%}  block  is  emitted  verbatim. 

•  Constant  definitions  are  converted  to  #define  macros. 

•  External  function  declarations  are  eliminated  since  these  functions  are  assumed  to  be 
declared  or  defined  within  the  code  included  via  the  7.7«{-  •  .7.7.}  block. 

•  The  number  of  nodes  (available  from  the  mapping  from  ids  to  roles)  is  defined  as  a  macro 
(line  18). 

•  A  new  variable  id  is  introduced  to  store  the  value  of  the  node’s  id.  The  value  of  this 
variable  is  supplied  from  the  command  line. 

•  Shared  variables  are  represented  using  containers  provided  by  the  MADARA  middle¬ 
ware  [12].  For  example  a  local  variable  of  type  unsigned  char  is  represented  as  a  madara 
variable  of  type  madara:  :UnsignedChar  (line  23).  Global  variables  are  expanded  to  ar¬ 
rays,  again  using  MADARA  containers.  For  example,  variables  formation  and  lock  are 
represented  as  1-dinrensional  (line  25)  and  3-dimensional  (line  26)  vectors,  respectively. 

•  A  function  (line  29)  is  generated  to  initialize  input  variables  and  the  node’s  id  from  com¬ 
mand  line  arguments.  Recall  that  whether  a  variable  is  an  input  or  not  may  depend  on 
the  role. 

•  A  function  (line  35)  is  generated  to  initialize  the  remaining  shared  variables. 

•  Internal  functions  defined  are  generated  (lines  42-43).  The  translation  from  the  dmpl 
function  is  syntax  directed.  However,  (i)  iterators  such  as  forall_node,  f orall_higher, 
exists_lower  etc.  are  appropriately  based  on  the  value  of  id  and  N;  and  (ii)  access  to 
global  variable  v  is  replaced  by  access  to  v  [id] .  Function  names  are  used  to  distinguish 
between  those  defined  at  the  top  level  and  those  defined  within  roles.  A  good  example  is 
REACHED_NEXT_XY  and  Leader_REACHED_NEXT_XY. 

•  A  MADARA  context  is  declared.  The  context  is  a  collection  of  variables  of  which  MADARA 
maintains  distributed  copies  on  each  node,  and  provides  the  consistency  required  by  the 
semantics  of  DMPL.  In  particular,  MADARA  attaches  a  Lamport  clock  with  each  variable 
v.  The  clock  is  initialized  to  0  and  incremented  with  each  assignment  to  u.  If  v  is  a  global 
variable,  and  it  is  assigned,  then  the  new  value  of  v  and  its  clock  are  sent  to  other  nodes 
via  network  messages.  Each  receiver  node  records  the  Lamport  clock  of  the  last  version 
of  v  received,  and  discards  a  received  value  of  v  if  it  comes  with  an  older  Lamport  clock. 
Note  that  this  implements  the  consistency  of  global  variables  required  by  the  semantics  of 
dmpl.  For  local  variables,  the  sequential  consistency  provided  by  the  hardware  directly 
implements  dmpl’s  semantics. 

•  Thread  functions  are  generated.  For  asynchronous  threads,  the  function  repeatedly  uses 
the  ZSRM  scheduler  (via  wait_f  or_next_period)  to  suspend  till  the  arrival  time  of  the 
next  job,  reads  the  shared  variables  atomically  (via  read_context),  executes  the  job, 
and  writes  the  shared  variables  back  atomically  (via  write_context).  Thread  functions 
are  named  based  on  whether  they  are  defined  globally  or  within  a  role.  A  good  exam¬ 
ple  of  an  asynchronous  thread  function  is  Leader_ADAPTATION_MANAGER.  The  functions 
read_context  and  write_context  use  a  special  mutex  provided  by  the  ZSRM  scheduler 
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1 

#include  <map> 

43 

void  REACHED_NEXT_XY  ()  {  ...  } 

2 

#include  " adaptation.manager .h" 

44 

void  Leader _REACHED_NEXT_XY  ()  { 

3 

double  accumulated_risk  =  0.0; 

45 

...  REACHED_NEXT_XY  ()  ;  ... 

4 

map < int , map < int , double >  >  hazard.map ; 

46 

> 

5 

47 

6 

void  INIT.HAZARDS (int  _X , int  _Y  , 

48 

madara :: Context  context;  //the  MADARA  context 

7 

int  tx  ,  int  ty)  {...}- 

49 

8 

double  GET_HAZARD ( int  x , int  y)  {...]- 

50 

/***  threads  ***/ 

9 

double  update_r isk ( double  risk) 

51 

void  C0LLISI0N_AV0IDANCE  ()  { 

10 

52 

int  s  =  0; 

11 

/***  constants  ***/ 

53 

for  (  ;  ;  )  { 

12 

#def ine  X  20 

54 

wait_for_next_period() ; 

13 

#def ine  Y  20 

55 

if(s  ==  0  &&  barrier ())  s  =  1; 

14 

#def ine  INITS  0 

56 

if(s  ==  1)  {  read.context () ;  s  =  2;  } 

15 

#def ine  NEXT  1 

57 

i f  ( s  ==  2)  -C 

16 

58 

//--  code  generated  from  body  of  thread 

17 

59 

//--  from  DMPL  program  goes  here 

18 

#define  N  5  //number  of  nodes 

60 

if ( ! j ob_missed_deadline  () )  s  =  3; 

19 

int  id;  //variable  to  store  node  id 

61 

} 

20 

62 

if(s  ==  3)  {  wr it e_cont ext ( ) ;  s  =  4 ;  } 

21 

/***  shared  variables  declaration  */ 

63 

if(s  ==  4  &&  barrier ())  s  =  0; 

22 

#include  <madara.h> 

64 

} 

23 

madara :: Uns ignedChar  state  =  INITS; 

65 

} 

24 

madara :: Uns ignedChar  x,  y,  xp ,  yp ,  xt ,  yt ; 

66 

25 

madara : : Vector<Bool>  formation(N) ; 

67 

void  Leader _ AD APT ATI 0N_ MANAGER  ( ) 

26 

madara : : Vector3<Bool>  lock(X,Y,N) ; 

68 

{ 

27 

69 

for  (  ;  ;  )  { 

28 

/**  get  input  variable  values  from  command  line 

*/ 

70 

wait_for_next_period() ; 

29 

void  initlnputsO  { 

71 

read_context  () ; 

30 

/*  Initialize  id,  x,  y,  formation.  Initialize 

72 

...  format ion  [id]  =  adaptat ion_manager  (  .  . 

31 

xt  ,  yt  if  id  corresponds  to  a  leader  role  . 

*/ 

73 

if (job_missed_deadline  ())  continue ; 

32 

} 

74 

writ e.cont ext  ()  ; 

33 

75 

} 

34 

/**  shared  variable  initialization  */ 

76 

} 

35 

void  initSharedQ 

77 

36 

{ 

78 

int  mainO 

37 

state  =  INITS;  xp  =  x;  yp  =  y; 

79 

{ 

38 

lock.set(x,y,id,l) ; 

80 

initlnputsO;  initShared  ()  ; 

39 

} 

81 

registerThreads  ()  ; 

40 

82 

startThreads  () ; 

41 

/***  internal  functions  ***/ 

83 

waitForThreads  () ;  return  0; 

42 

bool  NEXT.XY  ()  {  ...  > 

84 

} 

Figure  5.1:  Generated  C++  code  for  example  dmpl  program.  In  practice,  local  variables  (lines 
23-26)  are  duplicated  for  each  thread. 


to  protect  access  to  the  madara  context.  This  mutex  is  designed  to  prevent  priority- and- 
criticality  inversion  (i.e.,  a  higher  priority  thread  being  blocked  by  a  lower  priority  thread 
for  unbounded  time  on  trying  to  acquire  the  lock). 

•  For  synchronous  thread  functions  (e.g.,  C0LLISI0N_AV0IDANCE)  we  develop  and  use  a  real¬ 
time  implementation  of  the  2BSYNC  protocol  [4].  The  key  idea  behind  this  protocol  is  to 
use  two  barriers,  at  the  beginning  and  end  of  each  job  respectively,  across  all  the  nodes  to 
enforce  the  synchronous  model  of  computation.  However,  in  our  case,  since  threads  must 
respect  their  WCET  estimates,  a  barrier  cannot  be  implemented  by  an  unbounded  loop 
within  a  job  function.  Instead,  we  abort  a  job  whenever  the  barrier  condition  is  violated 
(lines  55  and  63)  and  retry  at  the  next  period.  Logically  this  is  equivalent  to  an  unbounded 
loop,  and  hence  the  correctness  of  our  implementation  follows  from  that  of  the  original 
2bsync  protocol  [4].  Note  also  that  we  use  the  ZSRM  scheduler  API  (line  60)  to  detect  if 
the  current  job  missed  its  deadline  due  to  overload,  and  abort  it  in  this  case. 

•  Finally,  the  main  function  (lines  80-83)  initializes  variables  from  command  line  arguments, 
initializes  remaining  shared  variables,  registers  all  threads  with  the  ZSRM  scheduler  using 
timing  parameters  specified  in  the  DMPL  file  and  the  Zero-Slack  Instants  computed  via 
ZSRM  schedulability  analysis,  starts  all  the  threads,  and  waits  for  them  to  complete.  In 
addition  to  the  threads  corresponding  to  the  role  specified  in  the  dmpl  program,  function 
registerThreads  (line  81)  also  registers  a  (high  priority)  MADARA  reader  thread  that 
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periodically  receives  new  global  variable  values  and  clocks  from  other  nodes  and  updates 
the  MADARA  context  appropriately. 
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6  Verification  via  Sequentialization 


In  this  section,  we  present  an  approach  for  verifying  the  reachability  properties  of  synchronous 
threads.  Consider  such  a  property  ip  =  (t,  /)  where  t  is  a  synchronous  thread  and  /  is  a  property 
function.  Since  t  respects  the  synchronous  model  of  computation,  it  can  be  shown  [4]  that  Pn 
is  over-approximated  by  a  sequential  program  that: 

1.  Maintains  two  copies  of  all  global  shared  variables  -  GV  q  and  GV i,  and  one  copy  of  local 
shared  variables  per  node  LV o, . . . ,  LV 

2.  For  each  DMPL  function  and  thread  /,  for  each  node  n,  produces  two  versions:  /b  and 
/£_.  For  each  n,  reads  from  GV o  and  writes  to  GV i,  while  function  reads  from 
GV i  and  writes  to  GV o-  Both  versions  read  and  write  the  same  set  of  local  variables.  A 
given  version  of  a  function  or  thread  calls  the  same  version  of  all  sub-functions. 

3.  Initializes  GV o,  GV i,  LVq,  . . . ,  LV i  as  per  constructors. 

4.  Executes  an  infinite  loop.  In  each  iteration,  it:  (i)  executes  the  thread  — »  version  of  t  once 
for  each  node,  i.e.,  n  times  with  the  node  id  set  to  0, 1, . . . ,  n  —  1;  (ii)  sets  the  values  of 
variables  that  can  be  modified  by  other  threads  non-deterministically  (to  over-approximate 
interference  from  them);  and  (iii)  repeats  (i),  but  with  the  -f—  version  of  all  functions. 

Informally,  GV 0  stores  the  values  of  global  variables  at  the  start  of  a  round;  GV \  stores  their 
values  at  the  end.  Thus,  every  program  instance  state  S(TT,t,i)  is  reachable  at  the  start  of 
the  i-th  iteration  of  the  loop  for  some  execution  of  this  sequential  program.  To  verify  whether 
< p  =  ( t ,  /)  holds,  we  can  prove  that  /  invoked  at  the  beginning  of  the  loop  always  returns  true. 


6.1  Implementing  Sequentialization 

Our  actual  implementation  of  sequentialization  has  the  following  features: 

•  Local  variables  are  expanded  to  arrays  so  we  can  represent  their  values  for  all  nodes.  This 
also  simplifies  generation  of  the  internal  and  thread  functions. 

•  Input  variables  are  initialized  non-deterministically.  However,  we  assume  that  the  property 
holds  at  startup.  The  assume  command  is  supported  in  some  form  by  most  state-of-the-art 
software  model  checkers.  It  semantically  cuts  off  all  executions  that  reach  the  command 
in  a  state  where  the  argument  to  the  command  evaluates  to  FALSE. 

•  This  program  is  written  in  the  C  programming  language  and  can  therefore  be  verified 
using  software  model  checkers  that  target  C  programs. 

6.2  Bug  Finding  and  Full  Verification 

For  bug  finding,  we  use  bounded  model  checking.  To  this  end,  we  change  the  infinite  loop  (Step  4 
above)  into  one  that  executes  k  times  for  some  target  bound  k.  We  verify  the  resulting  C  program 
with  a  software  model  checker.  For  complete  verification,  we  prove  that  the  property  /  (or  as 
is  typically  the  case,  a  strengthening  of  it)  is  inductive.  To  this  end,  we  create  a  program  that: 
(i)  initializes  all  variables  non-deterministically  but  assumes  that  /  returns  TRUE;  (ii)  executes 
the  body  of  the  loop  (Step  4  above)  once;  and  (iii)  asserts  that  /  returns  true.  We  then  verify 
this  program. 
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7  Evaluation 


We  have  implemented  a  compiler,  called  DMPLC,  that  performs  both  code  generation  and  se- 
quentialization  of  DMPL  programs.  The  size  of  the  generated  code  is  linear  in  the  size  of  the  input 
dmpl  program.  We  validated  DMPLC  on  five  examples  we  developed,  each  consisting  of  a  group 
of  robots  coordinating  to  achieve  some  goal.  We  used  the  V-REP  [29]  simulator  to  implement 
robots  and  their  movement  in  a  physical  environment.  In  particular,  we  used  quadcopter  models 
provided  by  V-REP  to  represent  each  node.  DMPLC  generates  appropriate  glue  code  to  use  the 
V-REP  API.  A  copy  of  DMPL,  the  examples,  and  instructions  for  generating  and  verifying  code 
are  available  at  https://db.tt/W153VDg6.  All  experiments  were  done  on  a  quad-core  2.90GHz 
laptop  with  16  gigabytes  of  RAM  running  Kubuntu  14.04  and  a  publicly-available  implementa¬ 
tion  of  the  ZSRM  schedulability  analysis  and  scheduler  [35].  The  DMPLC-generated  C-| — b  code 
was  compiled  with  g++. 


7.1  Reconnaissance  example 

We  implemented  the  five-node  reconnaissance  example  in  about  1300  lines  of  DMPL  code.  In  ad¬ 
dition,  we  used  an  external  adaptation  manager  component  that  was  implemented  in  about  1000 
lines  of  C++  code.  The  adaptation  technique  used  by  this  manager  is  described  elsewhere  [25] 
and  is  orthogonal  to  this  paper.  We  also  analyzed  collision  avoidance  in  two  ways: 

•  Bug  Finding:  First,  the  CBMC  [6]  tool  was  used  to  perform  bounded  model  checking 
of  the  sequentialized  code.  For  this,  the  assume  command  was  implemented  by  the 

_ CPR0VER_assume  function  supported  by  CBMC.  This  approach  is  incomplete,  but  was 

very  effective  for  finding  bugs.  We  found  several  bugs  due  to  programming  errors  (e.g., 
mistyped  variable  names,  missing  checks)  in  a  few  seconds  with  a  bound  of  5. 

•  Full  Verification:  Next  we  proved  correctness  by  showing  that  the  property  is  inductive 
over  each  execution  of  the  C0LLISI0N_AV0IDANCE  function.  Again  we  used  CBMC  for  model 
checking.  Since  the  collision  avoidance  property  is  not  inductive  by  itself,  we  manually 
strengthened  it  by  adding  extra  invariants,  and  were  able  to  prove  inductiveness.  In  all, 
10  new  invariants  were  added  over  a  1-person-day  effort,  and  verification  by  CBMC  at  the 
end  took  a  few  seconds. 


7.2  Other  examples 

We  implemented  several  other  examples,  including  the  following:  (i)  three  robots  moving  from 
predefined  starting  points  to  ending  points  on  a  grid  while  avoiding  collisions;  (ii)  five  robots 
moving  in  a  tight  formation  from  a  starting  to  an  ending  point  while  avoiding  collisions;  (iii)  nine 
robots  moving  in  a  loose  formation  from  a  starting  to  an  ending  point  while  avoiding  collisions; 
and  (iv)  nine  robots  moving  from  random  initial  positions  and  coming  together  at  the  grid  center 
to  form  a  3  x  3  square  structure.  In  each  example,  the  generated  code  demonstrated  expected 
behavior  during  simulation;  collision  avoidance  was  verified.  In  two  cases,  the  collision  avoidance 
implementation  initially  had  bugs.  (We  had  copied  the  DMPL  code  from  other  examples  that 
we  had  previously  verified,  but  subtle  differences  in  other  threads  were  unaccounted  for).  In  all 
cases,  our  approach  found  a  counter-example  that  we  used  to  find  and  fix  the  errors. 

Overall,  we  believe  dmpl  provides  a  good  tradeoff  between  programmability  and  verifiability  of 
DRTS.  Additional  examples  and  videos  are  available  at  the  DMPL  [11]  and  dart  [8]  web  sites. 
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8  Future  Work  and  Conclusion 


We  see  several  directions  for  ongoing  and  future  work:  (i)  developing  techniques  to  construct 
inductive  invariants  for  synchronous  threads;  (ii)  verification  techniques  for  liveness  properties, 
and  properties  over  asynchronous  threads;  (iii)  implementing  an  infrastructure  to  enforce  the 
isolation  between  code  generated  by  DMPL  and  external  functions;  currently  since  everything 
executes  in  the  same  process  and  the  external  functions  can  execute  arbitrary  code,  we  have  no 
way  to  ensure  this;  (iv)  handling  faults  and  intrusions;  this  is  important  especially  because  one 
promising  application  of  DRTS  is  in  autonomous  systems  where  human  intervention  is  limited; 
(v)  abstraction  and  compositional  verification  techniques. 

In  summary,  DRTS  is  a  promising  area  for  pushing  the  frontier  of  computer  science  and  engi¬ 
neering.  A  programming  language  that  has  well-defined  semantics  and  also  supports  verification 
is  critical  to  develop  liigh-assurance  software  for  such  systems.  The  language  must  deal  with 
timing  and  scliedulability  as  a  first  class  concern,  and  support  tasks  with  different  levels  of  crit¬ 
icality.  Finally,  to  facilitate  adoption,  it  must  play  well  with  existing  legacy  components  since 
very  few  DRTS  will  be  built  from  scratch.  While  by  no  means  the  final  answer,  we  believe  that 
dmpl  represents  a  promising  start  in  this  direction. 
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